{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 对全体数据，随机选择其中80%做训练数据，剩下20%为测试数据，评价指标为RMSE。（10分）\n",
    "说明：\n",
    "1) 实际上评价指标必须在训练完成后对结果评价时才能给出。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "训练数据有 584 笔，测试数据有 147 笔\n"
     ]
    }
   ],
   "source": [
    "# 数据读取及基本处理\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "# 绘图\n",
    "import seaborn as sn\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# 读取做完特征工程的文件FE_day.csv\n",
    "df = pd.read_csv('FE_day.csv')\n",
    "\n",
    "# 从原始数据中分离输入特征 X 和输出 y\n",
    "y = df['cnt']\n",
    "X = df.drop([\"cnt\"], axis = 1)\n",
    "\n",
    "# 特征名称，用于后续显示权重系数对应的特征\n",
    "feature_names = X.columns\n",
    "\n",
    "# 将数据分割为训练数据和测试数据\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# 随机采样 20% 的数据构建测试样本，其余 80% 作为训练样本\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=33, test_size=0.2)\n",
    "print('训练数据有 {} 笔，测试数据有 {} 笔'.format(X_train.shape[0],X_test.shape[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. 用训练数据训练最小二乘线性回归模型（20分）、岭回归模型、Lasso模型，其中岭回归模型（30分）和Lasso模型（30分），注意岭回归模型和Lasso模型的正则超参数调优。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 训练最小二乘线性回归模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>yr</td>\n",
       "      <td>4550.708897</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>temp</td>\n",
       "      <td>2654.792827</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>1287.992360</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>atemp</td>\n",
       "      <td>995.293778</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>929.887649</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>914.410909</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>season_4</td>\n",
       "      <td>830.579518</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>mnth_12</td>\n",
       "      <td>586.296405</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>mnth_8</td>\n",
       "      <td>517.803038</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>weathersit_2</td>\n",
       "      <td>409.589079</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>mnth_11</td>\n",
       "      <td>407.138803</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>238.417312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>workingday</td>\n",
       "      <td>216.956549</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>mnth_6</td>\n",
       "      <td>189.797216</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>season_2</td>\n",
       "      <td>85.141889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>weekday_5</td>\n",
       "      <td>77.516263</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>weekday_3</td>\n",
       "      <td>66.464787</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>weekday_4</td>\n",
       "      <td>43.650546</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>mnth_5</td>\n",
       "      <td>5.009200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>instant</td>\n",
       "      <td>-6.919060</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>weekday_2</td>\n",
       "      <td>-31.943212</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>season_3</td>\n",
       "      <td>-130.807162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>-191.612446</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>weekday_1</td>\n",
       "      <td>-202.493250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>-203.137582</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>holiday</td>\n",
       "      <td>-263.761415</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>mnth_4</td>\n",
       "      <td>-485.714239</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>mnth_3</td>\n",
       "      <td>-541.439917</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>season_1</td>\n",
       "      <td>-784.914245</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>-1174.845790</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>mnth_2</td>\n",
       "      <td>-1207.111892</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>hum</td>\n",
       "      <td>-1298.592138</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>-1323.999988</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>mnth_1</td>\n",
       "      <td>-1486.521042</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         columns         coef\n",
       "33            yr  4550.708897\n",
       "27          temp  2654.792827\n",
       "13        mnth_9  1287.992360\n",
       "28         atemp   995.293778\n",
       "14       mnth_10   929.887649\n",
       "17  weathersit_1   914.410909\n",
       "4       season_4   830.579518\n",
       "16       mnth_12   586.296405\n",
       "12        mnth_8   517.803038\n",
       "18  weathersit_2   409.589079\n",
       "15       mnth_11   407.138803\n",
       "26     weekday_6   238.417312\n",
       "32    workingday   216.956549\n",
       "10        mnth_6   189.797216\n",
       "2       season_2    85.141889\n",
       "25     weekday_5    77.516263\n",
       "23     weekday_3    66.464787\n",
       "24     weekday_4    43.650546\n",
       "9         mnth_5     5.009200\n",
       "0        instant    -6.919060\n",
       "22     weekday_2   -31.943212\n",
       "3       season_3  -130.807162\n",
       "20     weekday_0  -191.612446\n",
       "21     weekday_1  -202.493250\n",
       "11        mnth_7  -203.137582\n",
       "31       holiday  -263.761415\n",
       "8         mnth_4  -485.714239\n",
       "7         mnth_3  -541.439917\n",
       "1       season_1  -784.914245\n",
       "30     windspeed -1174.845790\n",
       "6         mnth_2 -1207.111892\n",
       "29           hum -1298.592138\n",
       "19  weathersit_3 -1323.999988\n",
       "5         mnth_1 -1486.521042"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 缺省参数的线性回归\n",
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "#1.使用默认配置初始化学习器实例\n",
    "lr = LinearRegression()\n",
    "\n",
    "#2.用训练数据训练模型参数\n",
    "lr.fit(X_train, y_train)\n",
    "\n",
    "#3.用训练好的模型对测试集进行预测\n",
    "y_test_pred_lr = lr.predict(X_test)  # 模型在测试集上的预测\n",
    "y_train_pred_lr = lr.predict(X_train)  # 模型在训练集上的预测\n",
    "\n",
    "# 看看各特征的权重系数(coef指系数)，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({'columns':list(feature_names), 'coef':list((lr.coef_.T))})\n",
    "fs.sort_values(by=['coef'], ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 最小二乘模型——模型评价"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The RMSE of LinearRegression on test is 814.4749076863649\n",
      "The r2 score of LinearRegression on test is 0.8279474225980328\n",
      "The RMSE of LinearRegression on train is 742.7543512758713\n",
      "The r2 score of LinearRegression on train is 0.8516480637403496\n"
     ]
    }
   ],
   "source": [
    "# 作业要求评价指标RMSE\n",
    "# 导入评价指标\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from sklearn.metrics import r2_score\n",
    "\n",
    "# 使用RMSE评价模型在测试机和训练集上的性能，并输出评估结果\n",
    "# 测试集\n",
    "print(\"The RMSE of LinearRegression on test is\", np.sqrt(mean_squared_error(y_test, y_test_pred_lr)))\n",
    "print(\"The r2 score of LinearRegression on test is\", r2_score(y_test, y_test_pred_lr))\n",
    "# 训练集\n",
    "print(\"The RMSE of LinearRegression on train is\", np.sqrt(mean_squared_error(y_train, y_train_pred_lr)))\n",
    "print(\"The r2 score of LinearRegression on train is\", r2_score(y_train, y_train_pred_lr))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 训练岭回归模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>temp</td>\n",
       "      <td>1778.493414</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>atemp</td>\n",
       "      <td>1546.374034</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>yr</td>\n",
       "      <td>1504.623924</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>914.843092</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>season_4</td>\n",
       "      <td>767.205317</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>678.352931</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>weathersit_2</td>\n",
       "      <td>388.865215</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>mnth_5</td>\n",
       "      <td>387.713628</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>mnth_6</td>\n",
       "      <td>369.663154</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>mnth_3</td>\n",
       "      <td>298.550050</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>227.515740</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>workingday</td>\n",
       "      <td>212.558710</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>mnth_8</td>\n",
       "      <td>205.686009</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>season_2</td>\n",
       "      <td>110.998614</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>mnth_4</td>\n",
       "      <td>106.306513</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>84.873020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>weekday_5</td>\n",
       "      <td>81.396550</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>weekday_4</td>\n",
       "      <td>62.795850</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>weekday_3</td>\n",
       "      <td>59.857933</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>instant</td>\n",
       "      <td>1.417157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>weekday_2</td>\n",
       "      <td>-34.140080</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>season_3</td>\n",
       "      <td>-91.388275</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>mnth_2</td>\n",
       "      <td>-143.125249</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>-191.851510</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>mnth_1</td>\n",
       "      <td>-194.714350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>weekday_1</td>\n",
       "      <td>-205.574484</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>-242.548582</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>holiday</td>\n",
       "      <td>-248.222941</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>mnth_11</td>\n",
       "      <td>-725.828529</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>season_1</td>\n",
       "      <td>-786.815656</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>mnth_12</td>\n",
       "      <td>-824.928596</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>-1087.773198</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>hum</td>\n",
       "      <td>-1142.397276</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>-1303.708307</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         columns         coef\n",
       "27          temp  1778.493414\n",
       "28         atemp  1546.374034\n",
       "33            yr  1504.623924\n",
       "17  weathersit_1   914.843092\n",
       "4       season_4   767.205317\n",
       "13        mnth_9   678.352931\n",
       "18  weathersit_2   388.865215\n",
       "9         mnth_5   387.713628\n",
       "10        mnth_6   369.663154\n",
       "7         mnth_3   298.550050\n",
       "26     weekday_6   227.515740\n",
       "32    workingday   212.558710\n",
       "12        mnth_8   205.686009\n",
       "2       season_2   110.998614\n",
       "8         mnth_4   106.306513\n",
       "14       mnth_10    84.873020\n",
       "25     weekday_5    81.396550\n",
       "24     weekday_4    62.795850\n",
       "23     weekday_3    59.857933\n",
       "0        instant     1.417157\n",
       "22     weekday_2   -34.140080\n",
       "3       season_3   -91.388275\n",
       "6         mnth_2  -143.125249\n",
       "20     weekday_0  -191.851510\n",
       "5         mnth_1  -194.714350\n",
       "21     weekday_1  -205.574484\n",
       "11        mnth_7  -242.548582\n",
       "31       holiday  -248.222941\n",
       "15       mnth_11  -725.828529\n",
       "1       season_1  -786.815656\n",
       "16       mnth_12  -824.928596\n",
       "30     windspeed -1087.773198\n",
       "29           hum -1142.397276\n",
       "19  weathersit_3 -1303.708307"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#岭回归／L2正则\n",
    "#class sklearn.linear_model.RidgeCV(alphas=(0.1, 1.0, 10.0), fit_intercept=True, \n",
    "#                                  normalize=False, scoring=None, cv=None, gcv_mode=None, \n",
    "#                                  store_cv_values=False)\n",
    "from sklearn.linear_model import  RidgeCV  # 带有广义交叉项的岭回归\n",
    "\n",
    "#1. 设置超参数（正则参数）范围\n",
    "alphas = [0.001, 0.01, 0.1, 1, 10, 100, 1000]\n",
    "\n",
    "#2. 生成一个RidgeCV实例\n",
    "ridge = RidgeCV(alphas=alphas, store_cv_values=True)\n",
    "\n",
    "#3. 模型训练\n",
    "ridge.fit(X_train, y_train)\n",
    "\n",
    "#4.用训练好的模型对测试集进行预测\n",
    "y_test_pred_ridge = ridge.predict(X_test)  # 模型在测试集上的预测\n",
    "y_train_pred_ridge = ridge.predict(X_train)  # 模型在训练集上的预测 \n",
    "\n",
    "# 看看各特征的权重系数(coef指系数)，系数的绝对值大小可视为该特征的重要性\n",
    "fs_2 = pd.DataFrame({'columns':list(feature_names), 'coef':list((ridge.coef_.T))})  # coef_为权重向量\n",
    "fs_2.sort_values(by=['coef'], ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 岭回归——模型评价"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The RMSE of RidgeCV on test is 812.0412685376801\n",
      "The r2 score of RidgeCV on test is 0.8289740676881535\n",
      "The RMSE of RidgeCV on train is 747.4410131599825\n",
      "The r2 score of RidgeCV on train is 0.8497700029725639\n"
     ]
    }
   ],
   "source": [
    "# 使用RMSE评价模型在测试机和训练集上的性能，并输出评估结果\n",
    "# 测试集\n",
    "print(\"The RMSE of RidgeCV on test is\", np.sqrt(mean_squared_error(y_test, y_test_pred_ridge)))\n",
    "print(\"The r2 score of RidgeCV on test is\", r2_score(y_test, y_test_pred_ridge))\n",
    "# 训练集\n",
    "print(\"The RMSE of RidgeCV on train is\", np.sqrt(mean_squared_error(y_train, y_train_pred_ridge)))\n",
    "print(\"The r2 score of RidgeCV on train is\", r2_score(y_train, y_train_pred_ridge))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 训练 Lasso 模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>yr</td>\n",
       "      <td>3856.665418</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>temp</td>\n",
       "      <td>2612.055688</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>1020.462105</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>atemp</td>\n",
       "      <td>1018.709568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>season_4</td>\n",
       "      <td>765.275311</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>595.292709</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>502.353082</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>mnth_8</td>\n",
       "      <td>312.017415</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>mnth_12</td>\n",
       "      <td>137.399521</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>mnth_6</td>\n",
       "      <td>106.418118</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>weekday_5</td>\n",
       "      <td>30.216101</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>workingday</td>\n",
       "      <td>25.747607</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>weekday_3</td>\n",
       "      <td>18.886198</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>mnth_11</td>\n",
       "      <td>10.088274</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>1.312455</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>season_2</td>\n",
       "      <td>0.364732</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>weekday_4</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>weathersit_2</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>instant</td>\n",
       "      <td>-5.020866</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>mnth_5</td>\n",
       "      <td>-15.048655</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>weekday_2</td>\n",
       "      <td>-78.604408</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>season_3</td>\n",
       "      <td>-200.973394</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>weekday_1</td>\n",
       "      <td>-248.678077</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>-349.043980</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>-427.799368</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>holiday</td>\n",
       "      <td>-446.994387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>mnth_4</td>\n",
       "      <td>-451.086432</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>mnth_3</td>\n",
       "      <td>-453.852559</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>season_1</td>\n",
       "      <td>-863.294182</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>mnth_2</td>\n",
       "      <td>-1068.288348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>-1175.860504</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>mnth_1</td>\n",
       "      <td>-1292.543807</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>hum</td>\n",
       "      <td>-1303.628880</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>-1730.891014</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         columns         coef\n",
       "33            yr  3856.665418\n",
       "27          temp  2612.055688\n",
       "13        mnth_9  1020.462105\n",
       "28         atemp  1018.709568\n",
       "4       season_4   765.275311\n",
       "14       mnth_10   595.292709\n",
       "17  weathersit_1   502.353082\n",
       "12        mnth_8   312.017415\n",
       "16       mnth_12   137.399521\n",
       "10        mnth_6   106.418118\n",
       "25     weekday_5    30.216101\n",
       "32    workingday    25.747607\n",
       "23     weekday_3    18.886198\n",
       "15       mnth_11    10.088274\n",
       "26     weekday_6     1.312455\n",
       "2       season_2     0.364732\n",
       "24     weekday_4    -0.000000\n",
       "18  weathersit_2     0.000000\n",
       "0        instant    -5.020866\n",
       "9         mnth_5   -15.048655\n",
       "22     weekday_2   -78.604408\n",
       "3       season_3  -200.973394\n",
       "21     weekday_1  -248.678077\n",
       "11        mnth_7  -349.043980\n",
       "20     weekday_0  -427.799368\n",
       "31       holiday  -446.994387\n",
       "8         mnth_4  -451.086432\n",
       "7         mnth_3  -453.852559\n",
       "1       season_1  -863.294182\n",
       "6         mnth_2 -1068.288348\n",
       "30     windspeed -1175.860504\n",
       "5         mnth_1 -1292.543807\n",
       "29           hum -1303.628880\n",
       "19  weathersit_3 -1730.891014"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#### Lasso／L1正则\n",
    "# class sklearn.linear_model.LassoCV(eps=0.001, n_alphas=100, alphas=None, fit_intercept=True, \n",
    "#                                    normalize=False, precompute=’auto’, max_iter=1000, \n",
    "#                                    tol=0.0001, copy_X=True, cv=None, verbose=False, n_jobs=1,\n",
    "#                                    positive=False, random_state=None, selection=’cyclic’)\n",
    "from sklearn.linear_model import LassoCV\n",
    "\n",
    "\n",
    "#1. 设置超参数（正则参数）范围\n",
    "alphas = [0.001, 0.01, 0.1, 1, 10, 100, 1000]\n",
    "\n",
    "#2. 生成一个LassoCV实例\n",
    "lasso = LassoCV(alphas=alphas, max_iter=10000)\n",
    "\n",
    "#3. 模型训练\n",
    "lasso.fit(X_train, y_train)\n",
    "\n",
    "#4.用训练好的模型对测试集进行预测\n",
    "y_test_pred_lasso = lasso.predict(X_test)  # 模型在测试集上的预测\n",
    "y_train_pred_lasso = lasso.predict(X_train)  # 模型在训练集上的预测 \n",
    "\n",
    "# 看看各特征的权重系数(coef指系数)，系数的绝对值大小可视为该特征的重要性\n",
    "fs_3 = pd.DataFrame({'columns':list(feature_names), 'coef':list((lasso.coef_.T))})\n",
    "fs_3.sort_values(by=['coef'], ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 讨论：\n",
    "1)只设定 alpha 值会出现大量类似的警告讯息:  \n",
    "C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\sklearn\\linear_model\\_coordinate_descent.py:472: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Duality gap: 126204783.36562662, tolerance: 176331.28855032122\n",
    "  tol, rng, random, positive)\n",
    "\n",
    "2)解决方法: 增加回圈的上限次数 (max_iter 缺省为 1000)为 10000次，  \n",
    "3)如果只是上调 tol (缺省为 0.0001)值，很可能导致计算的精确度不足, 无法得到最优解\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Lasso 模型评价"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The RMSE of LassoCV on test is 813.7897640845692\n",
      "The r2 score of LassoCV on test is 0.8282367651924001\n",
      "The RMSE of LassoCV on train is 742.9396650843979\n",
      "The r2 score of LassoCV on train is 0.8515740282485743\n"
     ]
    }
   ],
   "source": [
    "# 使用RMSE评价模型在测试机和训练集上的性能，并输出评估结果\n",
    "# 测试集\n",
    "print(\"The RMSE of LassoCV on test is\", np.sqrt(mean_squared_error(y_test, y_test_pred_lasso)))\n",
    "print(\"The r2 score of LassoCV on test is\", r2_score(y_test, y_test_pred_lasso))\n",
    "# 训练集\n",
    "print(\"The RMSE of LassoCV on train is\", np.sqrt(mean_squared_error(y_train, y_train_pred_lasso)))\n",
    "print(\"The r2 score of LassoCV on train is\", r2_score(y_train, y_train_pred_lasso))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. 比较用上述三种模型得到的各特征的系数，以及各模型在测试集上的性能。并简单说明原因。（10分）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The RMSE of LinearRegression on test is 814.4749076863649\n",
      "The RMSE of RidgeCV on test is 812.0412685376801\n",
      "The RMSE of LassoCV on test is 813.7897640845692\n",
      "The r2 score of LinearRegression on test is 0.8279474225980328\n",
      "The r2 score of RidgeCV on test is 0.8289740676881535\n",
      "The r2 score of LassoCV on test is 0.8282367651924001\n"
     ]
    }
   ],
   "source": [
    "print(\"The RMSE of LinearRegression on test is\", np.sqrt(mean_squared_error(y_test, y_test_pred_lr)))\n",
    "print(\"The RMSE of RidgeCV on test is\", np.sqrt(mean_squared_error(y_test, y_test_pred_ridge)))\n",
    "print(\"The RMSE of LassoCV on test is\", np.sqrt(mean_squared_error(y_test, y_test_pred_lasso)))\n",
    "\n",
    "print(\"The r2 score of LinearRegression on test is\", r2_score(y_test, y_test_pred_lr))\n",
    "print(\"The r2 score of RidgeCV on test is\", r2_score(y_test, y_test_pred_ridge))\n",
    "print(\"The r2 score of LassoCV on test is\", r2_score(y_test, y_test_pred_lasso))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过比较上述三种模型得到的特征系数我们可以看出，岭回归和 Lasso 回归都能使得线性回归系数收缩，并且在 Lasso 中有的特征系数为 0。回归系数都收缩是原因岭回归和 Lasso 都在最小二乘线性回归的基础上加了 L1 正则，限制了特征参数的取值，而 Lasso 中某些特征系数为 0，是因为对于 L1 正则，目标函数求的是次梯度，当梯度在次梯度集合内的时候，该维度的特征系数为 0.  \n",
    "  \n",
    "通过观察模型评价指标可以看出，在测试集上评价最好的是岭回归模型，其次是 Lasso 模型，最后是最小二乘线性回归。原因是岭回归和 Lasso 都在最小二乘线性回归模型中加入了正则项，防止了模型过拟合的问题，所以效果要更好些。而在特征分析中，我们看到有很多特征相关性比较大，比如说温度与体感温度，在特征多，且特征间存在共线性关系时使用 L2 正则效果要更好，所以这里岭回归模型比 Lasso 回归又更好一点点。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
